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Background / Context: 

Description of prior research and its intellectual context. 

Research on multi-tiered, research-based reading interventions provides strong evidence 
for the critical role of early reading instruction and the benefits of early intervention for children 
who are struggling to leam to read (Blachman et ah, 2004; Denton, Fletcher, Anthony, & 

Francis, 2006). Research on effective approaches for older students who have already 
experienced reading failure is less prevalent (Kamil et ah, 2008). In particular, there are few 
experimental studies documenting the effects of multi-tiered approaches to improving 
instructional outcomes in reading for students in the middle grades. To address this need, we 
have engaged a series of NIH-funded randomized studies on the effects of multi-tiered 
interventions for students with reading disabilities in grades six through eight. This poster 
presents selected findings from that ongoing work. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This poster extends previously reported research by addressing the question: “What are 
the effects of an intensive, small group, tutoring treatment in reading on the reading outcomes of 
students with significant reading disabilities who had little to no response to two previous years 
of intensive intervention?” 

Setting: 

Description of the research location. 

The research was conducted in 3 highly diverse middle schools in Austin and Houston, 
Texas, with half of the total sample drawn from each of these two communities. Students who 
received treatment participated in the intervention for one 45-50 minute period during their 
regular school day. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features or characteristics. 

Students who received treatment in Year 3 (also called Tier IV in this abstract) were 
those who 1) participated in intensive Tier II intervention in Year 1 of the study, 2) participated 
in an additional Tier II intervention in Year 2 (either standardized or individualized protocol), 
and 3) met the criteria for nonresponse in Years 1 and 2. Students met the end-of-year-year-one 
criteria for nonresponse if they scored less than 2100 scaled scored points (i.e., one-half of a 
standard error above the passing cut score) on the Texas Assessment of Knowledge and Skills 
(TAKS), or scored less than 90 standard scored points on the Woodcock Johnson III Letter- 
Word Identification assessment at posttest, or scored less than 90 standard scored points on the 
GRADE Comprehension Composite assessment at posttest (response status was determined only 
for students who regularly attended the treatment classes). The same criteria were used to 
detennine nonresponse at the end of year 2, with the exception that students must have scored 
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less than 2150 scaled scored points on the TAKS (one standard error above the cut score), 
instead of 2 100. Thirty students participated in the year 3 intervention. Students in the 
comparison group were randomly assigned to the “business as usual” condition in year 1 of the 
study (also called Tier I in this abstract) and met the same end-of-year criteria for nonresponse as 
treated students. In Year 3, there were 12 students in this comparison group. 

Of the students followed all three years of the intervention: 

• 46% were male and 54% were female, 

• 21% were identified as having limited English proficiency, 

• 43% were African American, 8% were Caucasian, and 49% were Hispanic, and 

• 86% received free or reduced lunch. 



Intervention / Program / Practice: 

Description of the intervention, program or practice, including details of administration and duration. 

During the first year of the study we provided a year-long, intensive intervention (over 
100 hours) to students in the treatment condition (see Vaughn et al., 2010). The intervention was 
implemented by tutors who were trained, coached, and supervised by the researchers. 
Instructional intensity was manipulated by controlling group size (i.e., small group instruction). 
In year 2, non-responding students in the treatment group were randomized to a standard or 
individualized protocol. Tutors were trained, coached, and supervised by the research team (see 
Vaughn et al., in review). The standard protocol was developed by the investigative team. The 
individualized protocol was less structured; tutors were provided with instructional options and 
coached on selecting appropriate strategies based on individual student need. The 3 rd year of 
treatment, or Tier IV, was an individualized intervention protocol implemented daily in 45-50 
minute periods for an entire year. This intervention incorporated the effective elements of a 
scientifically based reading intervention including very intensive instruction in word study, 
fluency, vocabulary, comprehension, text reading, and a motivation component. Teachers were 
allowed a degree of flexibility in responding to the instructional needs of individual students in 
their assigned groups. Teachers were also provided with intensive support in using data from 
curriculum based measures to assess mastery and plan ongoing instruction. Details on the 
treatment protocols for each year of the study are available at texasldcenter.org. 

Research Design: 

Description of research design ( e.g ., qualitative case study, quasi-experimental design, secondary > analysis, analytic 
essay, randomized field trial). 

The general design is a multi-gated, longitudinal, randomized trial with increasingly 
intense tiers of intervention as described earlier in this abstract. At-risk sixth-graders in year 1 of 
the study were included in the sample for this study. This poster presents findings from year 3, 
when the targeted cohort was in 8 th grade. While the “business as usual” group (Tier I) was 
detennined using the same end-of-year nonresponse criteria used with treatment condition and, 
as such, represent the best of available counterfactuals, remember that treatment condition 
students in year 3 of the study had the benefit of two prior years of intervention. In other words, 
students in both groups were low-responders based on the end-of-year criteria; however, 
participants in the treatment condition may have been advantaged prior to the year 3 study. 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 



The following measures were administered in year 3 of the study: 

• Woodcock-Johnson III Word-Attack and Letter-Word Identification (WJ-III; Woodcock, 
McGrew, & Mather, 2001); 

• Sight Word Efficiency and Phonemic Decoding Efficiency subtests from the Test of 
Word Reading Efficiency (TOWRE; Torgesen, Wagner, & Rashotte, 1999); 

• AIMSweb Reading Maze (Shinn & Shinn, 2002); 

• Test of Sentence Reading Efficiency (TOSRE; Wagner et al., 2010); 

• Gates MacGinitie Passage Comprehension; and 

• Passage Comprehension subtest of the Woodcock-Johnson III Tests (WJ-III; Woodcock, 
McGrew, & Mather, 2001). 

Psychometric infonnation on these measures is available at texasldcenter.org. 

Descriptive analyses were used to evaluate Year 3 pretest and posttest outcomes for the 
students in the intervention condition and for students in comparison group. Analysis of 
covariance was used to evaluate statistical significance and the Benjamini-Hochberg procedure 
was used to control for inflated Type I error. 

Findings / Results: 

Description of the main findings with specific details. 

Descriptive statistics are presented for the treatment (Tier IV) group in Table la and for 
the comparison (Tier I) group in Table lb for pretest and posttest. Absolute differences in pretest 
and posttest means for each measure and for each condition are provided; differences in pretest 
and posttest performance are also expressed in standardized units (Hedge’s g for repeated 
measures using the small sample correction) for each group. Measures are organized according 
to domain. Domains include: comprehension, fluency, and word reading accuracy. For the 
treatment group, there is a general pattern of improvement on all measures from pretest for 
posttest, with the exception of the Woodcock-Johnson III Letter-Word and Word-Attack 
subtests. In the comparison group, means at posttest are lower than pretest means, except on the 
Aimsweb Mazes measure, where there is a large increase from pretest to posttest. 

In the analyses of covariance, the pretest score was used as the covariate. There were 
statistically significant differences between treatment and comparison groups on the Gates- 
MacGinitie assessment, with adjusted posttest means of 18.52 and 12.95, respectively (F (\ , 37) 

= 10.543,/? = .002). Differences on the Woodcock Johnson III Letter-Word Identification 
subtest, (F( \ , 36) = 6.564,/? = .015) and on the TOWRE Phonemic Decoding subtest (F( \ , 36) = 
5.1 14,/? = .030) were also statistically significant. However, differences on the TOWRE 
Phonemic Decoding were not significant when Type I error rate was controlled using the 
Benjamini-Hochberg correction. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

While students in the Tier IV condition appear to benefit from year-3 treatment, they do 
not close the gap with the group of typically perfonning peers. Instead, the benefit derives 
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largely from the continuing downward trend of students in the Tier I condition compared to the 
relatively steady perfonnance of the students in the treatment group. The findings suggest that 
the most at-risk older readers may be able to maintain their status relative to their average age- 
mates when provided with intensive, daily intervention across the school year; however, these 
results do not indicate that treated students “catch up” to nonnative 8 th grade reading levels. 

The sample in this study is small, though not unusually so for studies of this type. The 
comparison group is a represents a reasonable counterfactual. However, given the longitudinal 
nature of the design and the fact that students in Tier 1 were initially randomized two years prior 
to the reported study, there may be some questions concerning internal validity. 
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Table la. Treatment (Tier IV) Student Outcomes 





Pretest 


Posttest 




Year 3 Measures 


Mean 


N 


Std. 

Deviation 


Mean 


N 


Std. 

Deviation 


Effect 

Size 


Gates MacGinitie Passage Comprehension 


17.39 


28 


5.3 


18.43 


28 


4.6 


0.18 


Woodcock Johnson III Passage 
Comprehension 


83.00 


28 


10.3 


84.39 


28 


9.5 


0.24 


Aimsweb Mazes 


171.75 


28 


48.8 


192.25 


28 


56.0 


0.38 


TOSRE 


71.50 


28 


6.5 


75.86 


28 


10.4 


0.46 


TOWRE Phonemic Decoding 


85.57 


28 


14.7 


87.07 


28 


14.7 


0.14 


TOWRE Sight Word 


89.93 


28 


9.8 


90.93 


28 


10.5 


0.10 


Woodcock Johnson III Letter-Word 
Identification 


89.39 


28 


11.6 


89.36 


28 


9.7 


-0.01 


Woodcock Johnson III Word Attack 


89.64 


28 


00 

00 


88.96 


28 


9.4 


-0.13 



Note: All measures are standard scores (M = 100, 5 = 15), except the Gates-MacGinitie Passage Comprehension and Aimsweb 
Mazes, which are indicated using raw scores. 



Table lb. Comparison (Tier I) Student Outcomes 





Pretest 


Posttest 




Year 3 Measures 


Mean 


N 


Std. 

Deviation 


Mean 


N 


Std. 

Deviation 


Effect 

Size 


Gates MacGinitie Passage Comprehension 


19.17 


12 


5.6 


13.17 


12 


5.7 


-0.69 


Woodcock Johnson III Passage 
Comprehension 


79.45 


11 


16.4 


78.91 


11 


11.9 


-0.08 


Aimsweb Mazes 


145.83 


12 


30.1 


179.17 


12 


50.5 


0.91 


TOSRE 


74.00 


12 


11.5 


72.33 


12 


13.8 


-0.18 


TOWRE Phonemic Decoding 


87.09 


11 


15.2 


80.91 


11 


12.6 


-0.96 


TOWRE Sight Word 


82.91 


11 


11.2 


82.45 


11 


9.8 


-0.07 


Woodcock Johnson III Letter-Word 
Identification 


86.36 


11 


14.9 


81.73 


11 


12.6 


-0.59 


Woodcock Johnson III Word Attack 


89.18 


11 


10.8 


85.36 


11 


12.3 


-0.48 



Note: All measures are standard scores (M= 100, 5 = 15), except the Gates-MacGinitie Passage Comprehension and Aimsweb 
Mazes, which are indicated using raw scores. 
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Table 2. ANCOVA Results 





Adjusted Posttest Means 








Year 3 Measures 


Tier IV 
(Treatment) 


Tier I 

(Comparison) 


/'-statistic 


/r-value 


partial ;/ 2 


Gates MacGinitie Passage 
Comprehension 


18.52 


12.95 


F(l, 37) = 10.543 


.002 


.222 


Woodcock Johnson III 
Passage Comprehension 


83.67 


80.76 


F(l, 36) = 2.499 


.123 


.065 


Aimsweb Mazes 


187.08 


191.24 


F(l, 37) = .063 


.804 


.002 


TOSRE 


76.48 


70.88 


F(l, 37) = 2.958 


.094 


.074 


TOWRE Phonemic 
Decoding 


87.39 


80.10 


F(l, 36) = 5.114 


.030* 


.124 


TOWRE Sight Word 


89.68 


85.62 


F(l, 36)= 1.743 


.195 


.046 


Woodcock Johnson III 
Letter-Word Identification 


88.76 


83.24 


F(l, 36) = 6.564 


.015 


.154 


Woodcock Johnson III 
Word Attack 


88.85 


85.66 


F(l, 36) = 2.209 


.136 


.058 



*Not significant with Benjamini-Hochberg Correction of the statistical significance of effects with multiple comparisons 
Note: All measures are standard scores (M= 100, 5 = 15), except the Gates-MacGinitie Passage Comprehension and Aimsweb 
Mazes, which are indicated using raw scores. 
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